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ABSTRACT 


The nain body of this report is divided into two parts. The 
first part presents a review and update of the discrete multivariate 
analysis techniques used for accuracy assessment. Appendix A con- 
tains a listing of the computer program written to implement these 
techniques. The second part presents new work on evaluating accuracy 
assessment using Monte Carlo simulation with different sampling 
schemes. 

Appendix B contains the results of the accuracy assessment 
analysis for the eight error matrices from the mapping effort of the 
San uuan National Forest. Appendix C contains a method of estimating 
the sample size reouirements ior implementing the accuracy assessment 
procedures. Appendix D contains a proposed method for determining 
the re 1 i ao i 1 i ty of change detection between two maps of the same area 
produced at different times. 
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1.0 Introduction 


This report 's divided into two parts. The first part deals 
with a short review and update of material described in last year's 
report (Congalton et aj_. 1981). This work involves assessing the 
accuracy of remotely sensed data using discrete multivariate analysis 
statistical techniques. 

The second part of tnis report describes the work currently 
in progress on sampling for accuracy assessment. This research is 
investigating different samoling schemes using Monte Carle simulation 
techniques. Although this work is not complete, some valuaDle results 
have already been achieved. 

2.0 Discrete Multivariate Analysis Techniques for Accuracy Assessment 

The three analysis procedures reviewed here all involve er’-o r 
matrices. An error matrix is a sauare array of numbers set out in 
rows and columns which express the number of cells assigned as a particular 
land cover type relative to tne actual cover type as verified in the field. 
The columns usually represent the reference data and the rows indicate 
either the Landsat classification or the photo interpretation. The 
discrete multivariate analysis procedures are perfonned on the error 
matrices. 


* 
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2.1 Review of the Normalization Procedure 

The first comparison procedure (Bishop et aK 1975) allows indi- 
vidual cell values in each error matrix to be compared. This comparison 
is mace possible by a process called normalizing the error matrix. 

This normal ization process is a way of standardizing each matrix so that 
a diroct comparison of individual cell values is possible. This procedure 
always converges to a unique set of maximum likelihood estimates and as 
such is the best algorithm to use in this case (Fienberg 1970). An 
assumotion made by this process is that all cells are of equal imoortance. 

Normalization of an error matrix is an iterative process dv wnicn 
the rows and columns of the matrix are successively balanced until each row 
and column add., up to a given value (marginal). This Drocess causes each 
cell value to be influenced by all the other cell values in its correspond- 
ing row and column. Each cell value is then a combination of reference 
data and remote sensor data and is represen; ative of both commission and 
omission errors for that land cover category. Because each row and column 
must add to a given marginal, the cell values in corresponding positions of 
two or more error matrices can then be compared without regard for differ- 
ences in sample size between matrices. 

The normalization process is performed by a computer program called 
MARGFIT (Conga 1 ton et 'J_. 1981). For additional details and samples of 
this process see Congalton (1981). 
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2.2 Update of the Test o* Agreement Procedure 


The second metnod of comparison is a procedure that tests for 
agreement between two or more error matrices (Bishop et al_. 1975). This 
measure of agreement is based on the difference between the actual 
agreement of the classification (i.e., agreement between remote sensor 
data and reference data indicated by the major diagonal) and the chance 
agreement which is indicated by the row and column marginals. This 
measure of agreement called KHAT is calculated by: 


n 


n 


k = 


1:1 (x ^ x ^ 

N 2 ' (*, + * * +1 ) 


where: 

n is the number of rows in the matrix 
x.. is the number of observations ir. row i and column i 
x^ + and x + . is the marginal total of row i and column i resoectively 
and N is the total number of observations. 


A KHAT value is calculated for each matrix and is a measure of 
how well the Landsat classification or photo interpretation agrees with 
the reference data. The approximate large sample variance of KHAT as 
determined by the delta method is: 
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where: 



Confidence intervals can be calculated for KHA7 using this ap- 
proximate large samde variance of KHA7. 7hese confidence intervals 
were used previously as a method for testing the significant difference 
between two error matrices. However, exact hypothesis tests are now 
availaLle and should be used instead of the confidence intervals. 

A test for significance of KHA7 can be Derformea to determine if 
the agreement between tne Landsat classification or onoto interoretation 
and reference data is significantly greater than zero. Also a test for 
the significant difference between two independent KHA7's can be Der- 
formed by evaluating the normal curve deviate (Cohen 1960). 7he test 
statistic for significant difference is approximately: 
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7he F0R7RAN computer program used to calculate this measure of 


agreement, KHA7, is called KAPPA. 7his program has been uDdated to 


fvj 
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include the exact hypothesis tests described above (Appendix A). Given 
the original matrix, the computer program implements a procedure that 
calculates the KHAT value and its corresponding variance. A confidence 
interval around K»AT is also computed along with the test statistic for 
significance of KHAT. All these values plus the values used in calcu- 
lating the variance (i.e, TH1 , TH2, TH3, and TH4) are printed out 
along with the original error matrix. The algorithm then confutes the 
test statistic for significant difference between independent KHAT'«: 
for eacn possible pair of matrices. These values are Drinted out in a 
summary table at the end of the program. 

2.3 Update of KAPPA Example 

As already mentioned, an actual test statistic is now available 
to test for significant differences between error matrices. In last 
year's report (Congalton et al . 1981) examoles were given in which 
only the confidence intervals were compared. Presented below is an 
updated analysis of the results that appeared last year in Table 6, 
page 19. This data compares four classification algorithms provided 
by Hoffer (1975). 


0 
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Table 1. Table of updated KHAT values. 


MATRIX 

KHAT 

VARIANCE 

COMPARISON 

Z STATISTIC 

Resu 

95 % 

Tt 

90 c * 

Nonsupervised 
(10 cluster) 
NS-10 

0.60479 

.00073735 

NS-10, NS- 
20 

0.47475 

NS 

NS 

Nonsupervised 
(20 cluster) 
NS-20 

0.58573 

.00087456 

NS-10, MS 

3.00930 

S 

s 

NS-10, MC 

-2.93550 

S 

! s 

Modi fed 
Supervi sed 
MS 

0.47581 

.00109972 ' 

| NS-20, MS 

2.47390 

s 


NS-20, MC 

-3.28090 ! S S 

1 : : 

Modified 

Clustering 

MC 

0.71846 

.00076218 

MS, MC 

1 -5.62360 
' 

i s ! s 

1 

/ 

I 1 

1 i 
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This analysis shows that there is not a significant difference 
between the classification obtained using a nonsupervised approach 
with 20 clusters and that obtained using a nonsupervised approach with 
10 clusters. However, the results of all the other tests yield 
significant differences between classification algorithms. 

2.4 Review of the Multi-factor Comparison Procedure 

The multi-factor comparison procedure allows more than one factor 
affecting the classification accuracy to be examined at the same time. 

The log-linear approach as described by Fienberg (1980) and BisnoD et al . 
(1975) is a method by whicn many variables and the interaction between 
these variables can be tested simultaneously to see which are necessary 
(i.e., significant) for explaining the classification accuracy. 

The simplest model (combination of variables) that provides a good 
fit to the data is chosen using a model selection procedure. This pro- 
cedure allows the user to systematically search all possible models and 
choose the simplest moael that provides a good fit to the data. First 
all uniform order models are tested (i.e., models with all possible n-way 
interactions, where n ranges from 1 to the number of factors) and the 
simplest good fit model is chosen. Each interaction of the chosen model 
is then tested for significance. If the interaction is not significant 
it is dropped until a model is found in which all the factors and 
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interactions of factors are significant. For a more detailed 

description of this stepwise model selection procedure, see Fienberg 

(1980) Section 5.3. The criteria for selecting a good model is based 

2 

on a Likelihood Ratio, G , and the degrees of freedom for the model. 

The Likelihood Ratio has an asymptotically chi-square distribution 
and therefore the critical value for testing if the model is a good 
fit can be obtained from a chi-square table using the appropriate 
degrees of freedom. 

The Likelihood Ratio is calculated using an Iterative Proportional 
Fitting procedure (Fienberg 1980 and Bishop et al_. 1975). This proceaure 
jses a method of successive approximations to converge to the maximum 
likelihood estimates of the minimum sufficient statistics as defined by 
the model. Therefore, the log-linear approach allows for analysis of 
multi-way tables with many factors. For example, error matrices generated 
using different dates, different algorithms, and different analysts all 
of the same scene of imagery can be put together and the factors necessary 
to explain the classification accuracy analyzed. This example would 
yield a five-way table with the five factors being: date, algorithm, 

analyst, Landsat classification, and reference data. Testing this five- 
way table would determine the simplest model of factors and interactions 
that best explain the results. 
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2.5 Multi-factor Comparison Example 

The data used to test the combined effects of different classi- 
fication algorithms and enhancement techniques on Landsat classifica- 
tion accuracy was supplied by Gregg et al. (1979). In this example 
two classification algorithms are performed on smoothed nnd unsmoothed 
imagery and the combined effects are studied. The factors and effects 
for this four-way table are listed in Table 2 and the original matrices 
presented in Table 3. Each algorithm classified the data into one 
of ten land cover categories (Table 4). 

A model selection procedure was performed on the four-way taDle 
beginning with the uniform order models (Table 5). The results of 
this procedure yields the simples best fit model to the data (Table 6). 
This model, [14] [24] [34], indicates that no three or four-way inter- 
actions are needed to explain the data. Instead, there are only two- 
way interactions involved. In other words, there is a combined effect 
due to each explanatory variable (i.e., algorithm, enhancement, and 
reference data) separately with the response variable. However, there 
are no higher order interactions. Therefore, each effect is important 
and no factor can be eliminated. The assumption that the error matrices 
adequately represent the actual classification must hold here if any of 
these results are to be meaningful. 






11 




















13 





















14 

TABLE 3 

( continued) 



S«« Tati# 4 


-ar a lij: 5 .' :>m 


lar.a cov*r . 


c 










<l- in 


A 3LZ 4 


■ of “he land cover categories for the four-wa 
comparing enhancement and classification metr 




























. 
















































3.0 Sampling Simulation for Accuracy Assessment 


This research involved a sampling simulation study using three 
deferent vegetation environments of varying spatial complexity. These 

three environments were forest, range, and agricultural lands. Small 

. 

areas (approximately 200 x 200 pixels) called subscenes were chosen from 
each of the three environments. Some of these subscenes contained large 

homogeneous areas of vegetation while others had very diverse vegetation. 

' 

Associated with each subscene were two classified data sets which were 
compared with each other to create a difference image. A difference 
image is a matrix of zeros and ones, where the zeros indicate agreement 
between the two data sets and the ones indicate disagreement. The popula- 
tion parameters were computed from a 100% sample (i.e., total enumeration) 
of the difference image. The difference image was also repeatedly sampled 
with various sampling schemes using Monte Carlo methods. A flow diagram 
of this procedure is displayed in Figure 1. 




3.1 Objectives 

The objectives of this research were to determine the best (minimum 
variance) unbiased sampling method to use on a given vegetation environment. 
This vegetation environment was then related to a pattern of classification 
error. Once the pattern of error was known, it was then possible to relate 
this sampling method to other areas of similar patterns of classification 


error. 


A reference data set 
or 

A Landsat classification 
(in digital format) 


' 




DIFIM 

computer program 


A Landsat classification 
(in digital format) 



Monte Carlo 
Methods 


Comparison and 
Results 






Figure 1. Flow diagram of sampling simulation procedure. 








3.2 Study Areas 

3.2.1 Forest Land Environment 

The forest land study area that was used in this project is the 
Lolo Creek area located in western Montana. The Bitteroot mountains are 
the dominant physical feature of the area with elevations ranging from 
3,000 feet to 9,200 feet. Average precipitation varies between 50 and 66 

centimeters per year. The vegetation of the area is characterized by 

1 

intermountain forest species. 

The subscene chosen for use in this project was the Garden Point 
7*i minute quadrangle. This subscene was classified using a 60 meter 
pixel and resulted in 12 land cover categories. Four of the land cover 
categories were roads while the other eight were vegetation types. 

3.2.2 Rangeland Environment 

The rangeland study that was used in this project is located in the 
northwest corner of Arizona in Mojave County. The area is approximately 
1,000,000 hectares in siza and is representative of a southwestern desert 
environment. The Colorado River is the major drainage tor the region. The 
area has a climate characterized by light precipitation, moderate tempera- 
tures, plentiful sunshine and low humidity. The vegetation varies from 
creosote bush and blackbrush at lower elevations to pinyon-juniper and 
ponderosa pine at higher elevations. The rangeland subscene chosen out of 
this study area was the Lizard Point minute quadrangle. This subscene 
was classified into nine land cover categories. The pixel size used here 
was 50 meters. 
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3.2.3 Agricultural Land Environment 

The agricultural land study area that was used in this project 
is the Umatilla Basin which occupies approximately 1.6 million acres 
in northcentral Oregon. This region is bounded to the north by the 
Columbia River. The area is characterized by an arid climate averaging 
less than 10 inches of precipitation per year. 

Center pivot irrigation is the major type of irrigation used in 
the northern section of the basis. It is in this area that a subscene 
was taken for study in this project. Data from the Clarke 7*s minute 
quadrange was available in Landsat classification and digitized reference 
data form. The classification was performed using a pixel size of one 
acre. 

3.3 Data 

At least one Landsat classification was available for each subscene. 
For the forest and range study areas two Landsat classifications were 
available. The forest study area had one classification performed using DMA 
terrain data with the Landsat data while the other classification used DEM 
terrain data along with Landsat. The range study area had one classification 
performed using DMA terrain data along with the Landsat data while the other 
classification was based on the Landsat data alone. 

The agriculture study area only had one Landsat classification. The 
other data set used was a reference data set derived from digitizing 
photography and land surveys. 


c 
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3.4 Procedure 

Once all the data was in digital format, a difference image 
was generated for each subscene using the computer program, DIFIM. 

This program processed the two corresponding data sets for each sub- 
scene pixel by pixel. When the two corresponding pixels were classi- 
fied the same, a zero was stored in that place in the output image. 

If the two corresponding pixels were classified differently, then a 
one was stored in that place in the output image. Therefore, an 
output image of zeros and ones was created and called the difference 
image. 

The difference image was then used to generate the population 
parameters for each subscene. Since the population (i.e., the sub- 
scene) was binomially distributed (i.e., a matrix of zeros and ones), 
the parameters of interest were the size of the population, N, the 
proportion of correct responses, P, and the variance. The population 
parameters were also calculated within the DIFIM program. 

After these calculations were completed each subscene was re- 
peatedly sampled using Monte Carlo methods and different sampling 
strategies. These sampling simulations were performed by a computer 
program cabled MCSAM. The required inputs for this program were the 
sampling scheme, the sample size, and the number of repetitions. The 
outputs of this program were the sample mean, sample variance, and the 
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number of times the population mean was not contained within the 
sample confidence interval. For cluster sampling the outputs also 
included a measure of relative efficiency and the intra-cluster 
correlation coefficient. 

3.5 Results 

As previously mentioned, all the results for this research have 
not been completed. Somp preliminary results are given below. 

3.5.1 Difference images 

The difference images created for each vegetation environment 
are in Figures 2-4. Note that the yellow shows the areas of agreement 
between the two data sets while the blue represents pixels of disagree- 
ment.- Also notice the patterns of error for each vegetation environ- 
ment. 

3.5.2 Intra-cluster correlation coefficients 

When using cluster sampling the effects of the cluster need to be 
measured. A measure of the homogeneity of the cluster is called ROH, 
intra-cluster correlation. The more homou »neous a cluster the greater 
the value of ROH. Intuitively one would like the cluster to be as 
diverse (i.e., heterogeneous) as possible so as to gain maximum information. 
Therefore it is desirable for ROH to approach zero. Figure 5 shows a plot 
of average ROH vs. cluster size for each of the vegetation environments. 
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figure 2 


L 



Difference image for agricultural environment. 
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Figure 3. Difference image for range environment. 




Figure 4. Difference image for forest environment 



AVERAGE ROH 
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3.6 Conclusions 

As expected the agriculture environment was the most homogeneous 
because of the large field sizes, while the forest environment was the 
most heterogeneous. The range environment had a mixture of large areas 
and small diverse areas and therefore fell somewhere between the 
agriculture and forest sites. These spatial patterns can be seen by 
looking at the difference images and also in the plot of ROH vs. cluster 
size. Remember that a large value of ROH (i.e., close to one) means tnat 
the cluster is more homogeneous. Therefore, as seen in Figure 5, the 
agriculture environment has the largest ROH while the forest site has 
the smallest. 

Also the plot of ROH vs. cluster size dictates some guidelines on 
what cluster sizes to use. Note that between 0 and 20 pixels/cluster 
ROH decreases ratner quickly while after arouna 20 oixels/ciuster tne 
improvement (i.e., decrease) in ROH occurs more slowly. This result 
dictates that large cluster sizes may not be gaining more information 
while costing more time and money to be researched. Therefore, despite 
the theoretical notion that ROH should be made to go to zero, it is more 
practical to use reasonable cluster sizes based on this plot and some 
economic information. 

3.7 Further Work 

There is a great deal of additional work to be done ir sampling 
simulation. This project has just begun and we hope to accomplish a 
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great deal more in the next year. Additional sampling schemes need 
to be investigated and new data sets collected. A possible new data 
set that contains both a Landsat cl assi f i cation and a reference map 
is a section of the San Juan National Fo est in Colorado. Further 
investigation in this area can lead to a Ivances in accuracy assessment 
procedures. 
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Appendix A 


FORTRAN COMPUTER PROGRAM KAPPA 


KAPPA 
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APPENDIX B 

RESULTS OF ACCURACY ASSESSMENT OF THE SAN JUAN 
N» "'AL FOREST R2MAP LAND COVER CLASSIFICATION 

The .. tic, k, was calculated for each of the eight 

matrices an / intract classification east and west (CCE, CCW), 
data base classif u-„ion east and west (DBE, DBW), abbreviated contract 
classification east and west (ACE, ACW), and abbreviated data base 
classification east and west (ADE, ADW). The resulting Kappa's, 
variances, and 95« confidence intervals for Kappaare displaced in 
Table B.l. The confidence intervals are displayed graphically in 
Figure B.l. Kappa was significantly greater than zero (a = .05) for 
all eastern classification, and was not significantly different from 
zero (a * .05) for all western classification. 

Classification accuracy as measured by Kappa was very low for 
all matrices. The low Kappa values, however, may not be entirely due 
to low classification accuracy. The large number of "NO SYMBOL" cate- 
gories in each matrix and the small sample sizes, particularly in the 
western matrices, also contribute to the low Kappa's. The extent of 
this contribution, however, cannot be assessed mathematically. 

Comparisons were made of the Kappa's for contract classification 
versus data base classification by location (east and west), for eastern 
classification versus western classification by classification type 
(contract and data base), and for abbreviated classification versus full 
classification by location (east and west). The results appear in 
Table B.2. Kappa's for the stated comparisons were not significantly 
different (a = .05) in any instance. 
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Further accuracy analyses of the matrices can be conducted to 
evaluate their usefulness for a particular purpose. For example, a 
weighting scheme can be used to emphasize categories of interest to 
wildlife managers while reducing the importance of forest and range 
categories. Another way to accomplish the same end without weights 
is to lump together categories whose value to a function is minimal. 

It Is very possible that these maps are quite suitable for one function 
while being inappropriate for another. These types of analyses can 
be performed quite easily and quickly if the necessary information 
(weights and/or categories to be lumped) is made available. 


40 


Table B.l. Kappa («), variance of Kappa, and 95% confidence interval 
for Kappa for each classification matrix. 


Matrix 

K 

o 2 

1C 

95% Confidence 
Interval-^ 

(Lower Limit, Upper Limit) 

CCE 

.149 

.00076 

.094, 

.203 

CCW 

.071 

.00281 

-.033, 

.175 

DBE 

.109 

.00070 

.058, 

.161 

DBU 

.108 

.00317 

-.002, 

.218 

ACE 

.153 

.00081 

.097, 

.208 

AC to 

.039 

.00312 

-.071, 

.148 

ADE 

.186 

.03260 

.122, 

.249 

ADW 

.u93 

.06179 

-.028, 

.214 


1 / See text for explanation of matrix identification abbreviations 
2/ Confidence interval calculated as k + 1.96o . 

— — K 


KAPPA 
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Figure B.l. Graphical representation of 95% confidence intervals 
for Kappa for each classification matrix. 



1/ See text for explanation of matrix identification abbreviations 


AL)W 
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Table B.2. Tests for significant differences between Kappa's 
from selected matrices. 


Matrices 

Z Value - 1 

Contract classification versus 

data base classification by location 

CCE vs. DBE 

1.02 

CCW vs. DBW 

0.48 

ACE vs. ACE 

0.76 

ACW vs. ADW 

0.65 


Eastern classificati on versus western classification 
by classification type 


CCE vs. CCW 

1.30 

DB£ vs. OBW 

0.02 

ACE vs. ACW 

1.32 

ADE vs. ADW 

1.32 

Abbreviated classification versus full 

classification by location 

CCE vs. ACE 

0.11 

CCW vs. ACW 

0.42 

DBE vs. ADE 

1.81 

DBW vs. ADW 

0.18 


J J See text for explar a .tion of matrix identification abbreviations. 

2 / Test statistic ^ - < A - < B ; Z must exceed 1.96 for the k's to be 

'rr—r 

*°kA °<b 

significantly different (a * .05). 
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APPENDIX C 

ESTIMATING SAMPLE SIZE REQUIREMENTS FOR 
ACCURACY ASSESSMENT USING KAPPA 

Sample size formula 

Sample size calculations for estimating Kappa are based on the 
confidence interval formula 

< + Z • a 

— K 

where * is Kappa, is the standard deviation of Kappa, and Z is a 
standard normal deviate. The value of Z may be selected uo yield an in- 
terval of the aesired confidence level. 

We will require the estimate of < to be within +E of the true k, 
where E is the allowable limit of error. This is equivalent to saying 

E = Zj . 

1C 

Since tne functional form of z is extremely complex, an approximate, 

< 

simpler form of will be used. This form is 

«« * Ao (1 - "c 1 

/ NO - P c ) 7 

wnere Pg is the actual agreement in the matrix, p £ is the chance agreement 
in the matrix, and N is the sample size. 
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The expression for E may now be rewritten as 

E = Z / V 1 " p p) 

/ N(1 - p c ) 2 

which leads to 

N = “(I 11 - ^ 

IwT" ' 

With selection of Z and E and estimation of p Q and p c tnis equation 

may be used to estimate sample size. This equation is equivalent to 

the statement: Unless a chance error has occurred, the chance being 

contro’led by Z, the estimate of < will be within +E of the true <. 

To implement the sample size equation p^ and p c must be 

estimated. The value p Q is the proportion of sample observations 

lyi’ig on the main diagonal of the matrix. This fraction may be 

est ; mated based on past analyses or an expected result. 

The value p is calculated as 
c 


t 



where t is the number of rows (and columns) in the matrix, p i+ is 
the proportion of observations assigned to category i by the classifi- 
cation algorithm, and p +1 - is the proportion of observations belonging 
to reference data category i. (The definitions of p. + and p +i may be 
switched by transposing the matrix.) 
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The proportion of observations in the i^ reference data category, 
p + ., may be estimateJ using the assumed proportion of land area in the 
coverage area that is in category i. While p^ + cannot usually be re- 
liably estimated prior to sampling, it is reasonable to assume that 
adequate classification would result in p i+ that is approximately equal 
to p +i - Therefore, p c can be rewritten as 

t 2 

p < * 

2 

These estimates of p n and p may be combined to estimate o . The 

U C K 

2 

given approximation for a ^ has been shown to be generally larger than 
the true variance, although this will not be true in every case. There- 
fore the calculated sample size, N, will generally be more than suf- 
ficient to attain the desired limit of error. 

Example sample size calculation 

Unless a 1 in 20 chance occurs, we wish to estimate < to within 
+.1. The area in question consists of only three distinct categories: 
Water (category 1); Forest (category 2); R?nge (category 3). The pro- 
portion of the area in each category is estimated to be: 


Category No. 

Jm. 

Proportion 

1 

Water 

.2 

2 

Forest 

.3 

3 

Range 

.5 
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In past analyses of the area approximately 605 of the observations 
fell on the main diagonal of the matrix. 

Now Z = 1.96 

E * .1 
P 0 • -6 

P c * (.2) 2 ♦ (.3) 2 + ( . S ) 2 * .38. 

Therefore 

N = - f • : 240 

(1 - .38 r (.1) 
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APPENDIX D 

A PROPOSED METHOD FOR ESTIMATING THE 
RELIABILITY OF CHANGE DETECTION 

Maps are often used to measure changes in cover types or land 
use over an interval of time. If the maps produced at each time were 
perfectly accurate, changes could be known without error. Most maps, 
however, contain errors in classification that make change detection 
subject to error. There are two perspectives from which to examine 
change; the first is a proportion of area basis, the second is a site 
specific basis. Change from a proportion of area perspective deals with 
the change in the proportion of an area assigned to a particular category 
over time. For example, a map made at time 1 identifies 50% of the 
covered area as water, while a map of the same area made at time 2 
identifies 60% of the area as water. No reference to a particular lo- 
cation is made, although the overall results can be applied to a 
particular location. 

Change from a site specific perspective deals with the change of 
a particular location from one category to another over time. Site 
specific change detection is extremely important in some rr.ip uses, but is 
more difficult to handle analytically than change in proportion to area. 
The following discussion will deal only with change from a proportion 
of area perspective. Hopefully, experience and insight gained in working 
with proportion of area change can be used to develop methods of dealing 
with site specific change. 


c 
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The method outlined below is only a first step in determining the 

reliability of change detection. Further research is necessary before 

this method is implemented operationally. 

Define two error matrices A and B, where matrix A is produced 

from mapping an area at time 1, matrix B is produced by mapping the same 

area at a subsequent time 2, and each matrix is comprised of the same 

A B 

categories. Let p i+ and p^ + denote the proportion of sample observations 

assigned to category i at times 1 and 2, respectively. 

A B 

Both p i+ and p. + are subject to errors of omission and commission 

A 

in the classification. If some measure of the reliability of p^ + and 

D 

P i+ could be determined, the reliability of the change over time coula 
be determined. 

The agreement measure Kappa, <, previously defined in this report, 
provides a type of reliability measure for an entire error matrix. 

A category specific measure of agreement, similar to Kappa, called 
is defined by Bishop et^ al_. . 1975, as 


= p ii " P i+ P fi 


p +i ‘ P i+ p +i 


where p.^ is the proportion of observations in the i th classification 
category and i th reference data category, p i+ is the proportion of 
observations in the i th reference data category, and p +1 - is the pro- 
portion of observations in the i th classification category. (The 
identity of the rows and columns can be exchanged by transposing 
the matrix.) has the same characteristics as < in that it accounts 
for both chance and actual agreement and has the same range. 
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0 


If interest is restricted to k . such that 0 <_ <_ 1 , then k . 

can serve as a reliability measure of a particular p i+ . It is reason- 
able to restrict attention to < ^ in this range since negative agreement 

In an error matrix is an undesirable and hopefully unusual occurrence. 

A B 

The reliability of the change from p i>+ to p^ + can now be calculated 

in the same manner as the reliability of a parallel circuit. In the 

A B 

parallel circuit the two components, in this case p i+ and p i + , have 

A B 

individual reliabilities, k . and , and the reliability of the entire 

A B 

circuit is calculated as • <°. This value can be calculated for each 
category resulting in a m?trix of reliabilities of changes from a 
category at time 1 to a category at time 2. 

The example in Figure D.l. shows the original error matrices A and 
B at times 1 and 2, respectively, their associated 's, and the matrix 
of reliabilities of change. This last matrix can be evaluated cell by 
cell, or an overall reliability can be found by averaging the cell entries. 
A further refinement in overall reliability calculation can be achieved 
by weighting each cell value by importance. 

Further research is needed to determine if can truly be 
interpreted as a reliability measure, what consequences must be accepted 
when some are less than zero, whether this method can be extended to 
cover unmatched categories between the two maps, and if the method can 
be used to determine site specific change reliability. 


.63 
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3 
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B 
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.48 

.71 


B 

B 

B 

*1 
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.71 
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.55 

.53 
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APPENDIX E 

Accuracy Assessment of the San Juan National 
Forest R2MAP Land Cover Classification 

E.l Introduction 

An accuracy assessment was conducted for the San Juan National 
Forest. Specifically, this included development of error matrices 
to supplement the preliminary evaluation made by Lockheed Electronics 
Company,* Inc. Lockheed has used Landsat digital data to map land 

cover/ vegetation for the entire San Juan National Forest from two 

adjacent scenes according to a classification system agreed to by the 
Forest Service. Personnel at Virginia Tech worked with managers on 
the San Juan National Forest to determine the accuracy for (a) the 
east half of the Forest (from the eastern Landsat scene); (b) the 

west half of the Forest (from the western Landsat scene); (c) using 

the classification system agreed to in the Lockheed contract; and (d) 
according to the classification system used in development of the 
Forest's "R2MAP" dioital data base. An explanation for the two classifi- 
cation systems is given in Appendix F. Also the corresponding R2MAP 
symbols are given in Tables E.l and E.2. 

E.2 Procedure 

The following procedure was used to conduct the accuracy 
assessment: 

* Mazade, A. V. , C. A. Underwood, J. F. Ward, and S. S. Vao. 1979. 

Remote Sensing and Computer-Based Vegetation Mapping in the San 
Juan National Forest, Colorado. Final Report LEC-13792, Lockheed 
Electronics Co., Inc. 60 pp. 
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Table E.l. The contract land cover categories and the corresponding 
R2MAP symbols. 


Land Cover 
Categories 


Symbol 


West half of 
forest 


East half of 
forest 


Aspen/Cottonwood 

G 

95 A4 

$2 A3 94 A4 

Aspen/Conifer 

BB 

No R2MAP Symbol 

E2 

Ponderosa Pine 

A 

X5 

PI P3 

Spruce- FI r 

E 

No R2MAP Symbol 

S4 

Douglas-Fir 

C 

No R2MAP Symbol 

C4 D3 

Ponderosa Pine/Oak 

7 

im 

F2 F3 

No R2MAP Symbol 

Conifer/ As pen 

AA 

No R2MAP Symbol 

H4 

Oak 

K 

01 04 02 03 L2 

04 03 02 

Pinyon/ Juniper 

M 

J4 

J1 02 &1 

Oak/Conifer 

CC 

T3 

Z5 

Rock/Barren 

U 

Y5 05 /5 A5 
V5 =5 ]5 05 A5 N5 

X5 R5 

Willow 

0 

No R2MAP Symbol 

]5 

Mixed Brush 

Q 

B4 B3 

No R2MAP Symbol 

Mesic 

R 

M5 

No R2MAP Symbol 

Grass 

Y 

Z5 G5 <.5 

G5 =5 /5 

Alpine 

X 

No R2MAP Symbol 

M5 :5 

Rocky/Grass 

DD 

6 

15 55 05 

Water 

V 

W5 

W5 #5 

Other 

W 

U5 55 #5 

05 <5 
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Table E.2. The data base land cover categories and the corresponding 
R2MAP symbols. 


Land Cover 
Categories 

Symbol 

West half of 
forest 


East half 
forest 

Cottonwood (>30%) 

I 

No symbol 

No 

symbol 

Aspen (>30%) . 

6 

95 A4 

$2 

A3 94 A4 

Ponderosa Pine (>30%) 

A 

F3 

PI 

P3 

Spruce-Fir f >30%) 

E 

No symbol 

S4 


Douglas-Fir (>30%) 

C 

No symbol 

C4 

D3 

Oak (>30%) 

K 

04 03 02 T3 

04 

03 02 

Pinyon Juniper (>30%) 

M 

J4 

No 

symbol 

Cottonwood (10-30%) 

J 

No symbol 

]5 


Aspen (10-30%) 

H 

No symbol 

E2 


Ponderosa Pine (10-30%) 

B 

X5 F2 

Z5 


Spruce- Fir (10-30%) 

F 

No symbol 

No 

symbol 

Douglas-Fir (10-30%) 

D 

No symbol 

No 

symbol 

Oak (10-30%) 

L 

L2 01 

No 

symbol 

Pinyon Juniper (10-30%) 

N 

No symbol 

J1 

02 &1 

Rock/Barren 

U 

V5 =5 ]5 @5 
A5 N5 Y5 05 R5 /5 

X5 

R5 

Willow 

0 

No symbol 

No 

symbol 

Mixed Brush 

p 

B4 B3 

No 

symbol 

Sage 

Q 

No symbol 

No 

symbol 

Meadow 

R 

M5 

M5 


Sonoran 

S 

G5 

05 
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Table E.2. (Continued) 

Land Cover 

Categories Symbol 

West half of 
forest 

East half 
forest 

Montane 

T 

Z5 <5 6 

G5 =5 

Alpine Xeric 

Y 

No symbol 

15 55 :5 

Alpine Mesic 

X 

No symbol 

/5 -5 

Water 

V 

W5 

W5 #5 

Other 

W 

U5 55 #5 

@5 <5 
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Step I . 

The staff of the San Juan National Forest visited many areas on 
the ground and related them to the symbols printed on the R2MAPS . This 
permitted the local resource managers to develop a detailed definition 
of the Landsat classification categories. Also, the field crews became 
familiar with characteristics of each category as they appear on color 
infrared aerial photography. A set of "photo examples" for each category 
was made for use by all the photo interpreters. This should help assure 
consistency in the ground reference data collection. 

Step II . 

Computer printouts which summarize the ni.mber of acres of each 
Landsat category classified on each quad (from R2MAP) was produced. The 
forest boundary was used to screen only those R2MAP cells within the 
Natioral Forest. All private lands within the forest were delted. This 
permitted the forest to be stratified into the various Landsat categories 
whicn were each sampled proportional ly . (Note that the classification 
was sampled and not the ground cover.) After the re"!ative proportions 
of each strata were determined the number of pixels (3 ac. cells) within 
each quad that should be sampled by each category were computed. 

Step III . 

A list of random coordinates were compiled for use in selecting 
sample cells within each individual quad. Cells were sampled without 



56 


replacement until the desired number of cells needed for each category 
was reached. The location of each cell kept and used in the evaluation 
were transferred to its corresponding location on the topographic map 
and assigned a sample number. 

Step IV . 

Each R2MAP cell selected was next transferred from the topographic 
map to the 1:30,000 scale 9 x 18 inch aerial photography (flown in 
September, 1981) and delineated on a transparent overlay fastened to the 
photo. No indication cf how Landsat classified each cell was put on 
the overlay. (This would bias the photo interpretation.) Only the 
sample lumber was next to each cell. 

Step V . 

Three independent, photo interpreters assigned a Landsat category to 
each sample block according to the category definitions and "photo 
examples" developed in step I Complete interpretation agreement among the 
three interpreters was mandatory. All differences in category assignment 
were resolved. This required the pnoto interpreters to meet and "negotiate" 
a proper interpretation. 

Step VI . 

Virginia Tech designed a technique and administered a test to assure 
that consistent photo interpretation was achieved. Also, Virginia Tech 
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designed tne forms for recording all data, reviewed the category 
definitions and all procedures. Finally, Virginia Tech compiled and 
reported the final error matrix for the R2MAP accuracy. The data 
were comoiled so that one matrix was produced for the east portion 
(i.e., east Landsat scene) and one for the west (i.e., west Landsat 
scene) under each classification system. These matrices are given 
in Tables E.3, E.4, E.5, E.6, E.7, E.8, E.9, and E.10. Note that 
there were several instances where there was no R2MAP symbol which 
corresponds to the land cover categories under both the contract or 
data Dase classification systems. 


E.3 Sample Size Determinat 'on 

Map accuracy was aetermined using the K statistic (Bishop, 
Fienbera, and Holland, 1575): 
n n 


K = 


N 

N 2 - £ x, + x +i 


where 

N = number of observations, 
n = number of categories, 

= number of observations c’assified as category i by both 

photo interpretation and the Landsat classification algorithm, 
x^+ = number of observations classified as category i by photo 
interpretation, 

x +i * number of observations classified as category i by the Landsat 
classification algorithm. 
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Table E.3. Contrast Classification - East Half. 


REFERENCE DATA (PHOTO INTERPRETATION] 
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TaMe E 5. Abbreviated Contract Classification - 
East Half. 


REFERENCE DATA 
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Table E.6. Abbreviated Contract Classification - 
West Half. 


REFERENCE DATA 
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Table E.8. Dat: Base Classification - West Half. 



















CLASSIFICATION (R2 MAP SYMBOL ) 
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Table E.9. Abbreviated Data Base Classification - 
East Half. 
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The null hypothesis of K equal to zero can be tested statistic- 

- *2 

ally using the asymptotic variance. The asymptotic variance of K, a 
(K), is available, but was not stated here because of its complexity. 

A O A 

The factors affecting the size of cr ( K) that can be controlled 
by sampling are the number of observations, N, and the number of cate- 
gories, n. Since the number of categories is set by map requirements , 
only the number of observations can be controlled. 

A small presample was gathered to provide information about the 
size of the variance for a given sample size in this particular situation. 

A final sample size for determining overall map accuracy can then be 
selected using the presample information as a base. 

The sources of agreement and disagreement between classification by 
photo interpretation and by Landsat algorithm can be investigated using 
the techniques of categorical data analysis (Bishop, Fienberg, and Holland, 
1975). Sample sizes necessary for these techniques are fixed by the 
analysis method rather than by the degree of precision desired; sample 
sizes below a certain threshhold are simply too low to allow analysis. 

The usual sample size required to perform categorical data analysis 
is five times the square of the number of categories. This sample size is 
expected to be considerably larger than that required to determine overall 
map accuracy, but the larger sample size is required if the sources of 
error in the map are to be identified. 


67 


E.4 Summary 

The number of samples which were taken (due to personnel time) 
for the accuracy assessment was too small to give reliable results at 
any specified precision level. However, it is clear that the R2MAP 
data (i.e., 3 acre cell category labels) are quite different from the 
consensus of the three photo interpreters. These errors may in large part 
be due to misregistration in the Landsat classification. Also, some error 
could be attributed to the process of resampling the 1 acre Landsat pixels 
to form the 3 acre R2MAP cell classifications. Analyses of this 
assessment are given in Appendix B. 

The utility of the land cover data in R2MAP for use by the San Juan 
National Forest will have to be judged by the Forest Service personnel. 
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APPENDIX F 

Explanation of the Two Land Cover Classification 
Systems used in the San Juan National Forest 
Accuracy Assessment 
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Attachmenc 2 

SAN JUAN NATIONAL FOREST 
REMOTE SENSING PROJECT 

Contract Classification of Cover Classes 
for Accuracy Assessment 

The following classification system further defines the cover types for 
which the R2MAP symbols were developed. The original definitions in Exhibit 
A, p.A-4 of the Remote Sensing and Computer Base d Vegetation Mapp lng in the 
San Juan National F orest, Colorado . Final Report for USDA/FS Contract 53- 
82x9-8-2338 October 11, 1978 - September 1, 1979, were followed as closely 
as possible to maintain as much consistency as possible with the work already 
completed. The primary problems with the existing definitions was their 
tendency to overlap. The cover type key presented here will reduce this " 
tendency . 

The attached diagram provides a schematic view of the key. These cover 
classes and cover types apply to 3 acre cells which are the basic unit in 
R2MAP. 


The second attached key is for the Forest data Lase. 


COVER TYPE DIVISIONS 


FOREST - Cells winh crown cover > 302 

- 2 refers to 2 of existing crown or ground cover in a 3 acre cell. 





7e f f"C. 


30% 


-/I - f\ n* / Oo K I 

' 70 7o 


NON FOREST - Cells with-i 302 


- 2 refers to 2 of the cell not covered by tree 1 crown cover 


0% to n 


J3ruZ h 


'«~7. 


Note: Within the forested cover types, species dominance drives the classification 

system. Between forest and nonforest cover types and within the nonforest types 
a hierarchical system exists. Forest types override nonforest types. Brush over- 
rides grass types. 
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SAil JUAN* NATIONAL FOREST 
Ground "over Type Kev- Contract 
(Cove- Types ar* underlined) 


I. 

Lands 

IE. 

Non forest 

LA. 

Forest 

IB1 . 

Rock Barren 

LAI. 

Commercial 

IB2. 

Brush 

IA1(4). 

Hardwood 

IB2(a) . 

Willow 

IA1 (a)(1). 

Asoeu/ Cottonwood 

152 (b). 

Mixed Brush (Sa?e 

La 1 , (a) (2) . 

Aspen/ Conifer 

IB3. 

Grass 

IA1 (b). 

Conifer 

133(a) . 

Mesic (Meadows) 

Lt»i(b) (1) . 

Ponderosa Pine 

IB3(b) . 

Grass (Sonoran & Montane 

IA1 (b)(2). 

Spruce-Fir 

IB3(c) . 

Alpine (Xeric A Mesic) 

IAI (b)(3). 

Dou?!as-fir 

133(d). 

Rockv/Grass 

IA1 (b)(4). 

Ponderosa Pine/bak 


• 

IA1 (b)(5). 

Conif erX\spen 

• 


IA2. 

Non Come T ' \al Forest 

II. 

Water 

IA2(a). 

Oak 

III. 

Snow 

IA2(p). 

1 

Pinon/Juuip*.r 

IV. 

Shadow 

Ia2(c) . 

Oak /Conifer 




SAN JUAN NATIONAL FOREST 
Cround Cover Type Key - Contract 
(Cover types are underlined) 
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Saa Juan National Forest 
Ground Cover Type Key - Contract 

The objective of this key is to classify the total area within the San 
Juan National Forest into one of 18 cover types by three acre cells. 

I. LAND - Cells covered by > 501 land. 

Yes - Go to I. A. 

No - Go to II. 

I. A. FOREST - Cells covered by tree species >301 crown cover. (Trees are 
further defined as woody vegetation capable of producing a woody stem 
2.12 feet in height. This includes oak and other tree species that**; 
<12 feet in height due to site limiting conditions.) 

Yes - Go to I. A. 1. or 2. 

No - Go to I.B. 

I. A.l. COMMERCIAL FOREST - >501 of the crown cover is one or more of 
the following commercial species: ponderosa pine, spruce-fir, 

Douglas-fir, aspen, or cottonwood. 

Yes - Go to I. A. 1. (a) or (b) 

No - Go to I. A. 2. 

I. A.l. (a) HARDWOOD - >50* of the crown cover is one or more of the 
following hardwood species: Aspen or Cottonwood. 

Yes - Go to I. A. 1. (a) (1) or (2) 

No - Go to I. A. 1. (b) 

I. A.i. (a) (1) ASPEN/ COTTONWOOD - >70, of the crown cover is 

Aspen or Cottonwood. 

Yes - The cover type is Aspen/Cottonwood . 

No - Go to I. A. 1. (a) (2). 


I. A.l. (a) 

•/ 


(2) ASPEN/CONIFER - Aspen crown cover is 7-50% but not 

7 701. The conifer crown cover is <501 but not <301. 


Yes - The cover type is Aspen/Conifer . 


I. A.l. (b) CONIFER ->501 of the crown cover is one or more of the 

following conifer species: ponderosa pine, spruce-fir, or 

Douglas-fir. 


Ye.« - Go to I.A.l. (b) (1), (2), (3), (4) or (5). 
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I. A.l. (b) (1) PONDEROSA PINE - >701 of the crown cover is ponderosa 

pine. 

Yes - The cover type is ponderosa pine. 

No - Go to I. A.l. (b) (2), (3), (0, or (5). 

I. A.l. (b) (2) SPRUCE-FIR - >70% of the crown cover is mixed spruce 
and fir. 


I. A.l. (b) 


Yes - The cover type is spruce-fir. 

No - Go to I. A. 1. (b) (3), W^r (5). 

(3) DOUGLAS *FIR - >70% of the crown cover is mixed 

Douglas -fir and white fir. 


Yes - The cover type is Douglas-fir . 
No-Go to I. A. 1. (4) or (5). 


I. A.l. (b) (4) 


PONDEROSA PINF./OAK - Ponderosa pine crown cover is 
>50% but not >70%. The oak crown cover is < 50% but 
not < 30%. 


Yes - The cover tvne is oonderosa Dine/oak. 
No - Go to I.A. 1." ~(b) (.5 J . 


I. A.l. (b) (5) 


CONIFER/ ASPEN - Conifer crown cover is > 50% but 
not > 70%. The aspen crown cover is<5C% but not 
< 30%. 


Yes - The cover type is Conifer/Aspen . 


I. A. 2. NONCOMMERCIAL FOREST ->50% of the crown cover is one or more 
of the following noncommercial species: pinon pine, juniper, 

or oak. 


Yes - Go to I. A. 2. (a) (b) or (c) 

I. A. 2. (a) OAK ->70% of the crown cover present is oak. 

Yes - The ground cover type is oak. 

No - Go to I. A. 2. (b) or (c). 

I. A. 2. (b) PINON/JUNIPER ->50% of the crown cover present is pinon/ 
junif :r. 

Yes - The ground cover type is pinon/ juniper. 

No - Go to I.A.2. (c). 

I. A. 2. (c) OAK/CONIFER - Oak crown cover is >50%, but not >70%. The conife: 
crown cover is < 50% but not <30%. 

Yes - The ground cover type is oak/conifer. 
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I. B. NONFOREST - Cells covered by <301 crown cover of tree species 
Yes • Go to- I.B. , 1. 2. or 3. 

I. B.l. ROCK/BARREN -<30% vegetative ground cover is present. 

Yes - Cover type is Rock/Barren. 

No - Go to I.B. 2 or 3. 

I. B.2. BRUSH - >301 of the area is covered by brush species. 

Yes - Go to I.B. 2. (a) or (b) 

No - Go to I.B. 3. 

I. B.2. (a) WILLOW (Brush) - >30% willow crown cover is present. 

Yes - The cover type is Willow. 

No - Go to I.B. 2. (b). 


I. B.2. (b) MIXED BRUSH (Sage) - or. the area is brush other chin 

oakbrush or willows. - 

Yes - The cover type ; s Mix-. 1 Brush (Sage). 

I. B.3. GRASS ->30% of the area is gr»ss and herbaceous plants. 

Yes - Go to I.B. 3. (a) (b) (c) or (d) . 

I. B.3. (a) WET or MESIC GRASSLAND - The area is dommat.d by grasses 
and other herbaceous plants requiring constant water 
availability. The elevation range for this cover type 
is 6,500 feet to 11,000 feet. 

Yes - The cover type is Wet or Mesic Grassland . 

No - Go to I. B. 3. (b), (c) or (d). 

I. 3.3. (b) GRASSLAND (Scnoran and Mountain) - The area is dominated by 
grasses and other herbaceous plants. The elevation range is 
6,500 feet to 11,000 feet. 

Yes - The cover tvne is Grassland. 

No - Go to I.B. 3. (c) or (d). 

I. B.3. (c) ALPINE (Xeric jnd Mesic) - The area is above timberline 
and dominated by grass and other herbaceous plants. The 
elevation range of this cover type is > 11,000 feet. 


Yes - The cover type is ALPINE (Xeric and Mesic). 
No - Go to I.B. 3. (d). 
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I. B.3. (d) ROCKY/ GRASSLAND - The area is dominated by grasses and 
other herbaceous species. Rock and barren soil cover 
>301 but <50* of the area. The elevation range of the 
area is ^6,500. 

Yes - The cover type is Rocky/Grassland . 

II. WATER - Cells covered by >501 water. 

Yes - Ground cover type is water . 

III. SNOW - This cover class division is included because areas will seasonally 
be covered with snow. Landsat will record this information, if present. 
Snow is not a valid ground cover type. 

IV. SHADOW - This cover class division ij included because steep topography 
on the Forest produces shaded cells regardless of sun angle. Landsat will 
record this information, if present. Shadow is not a valid ground cover 
type. 
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San Juan National Forest 
Ground Cover Type Key - Data 3asa 
(Cover types are underlined. ) 


z. 

Lands 

I. A. 

Forest ( > 102 stocked) 

I.A.l. 

Forest ( 7 302) 

I. A. l.a. 

Commercial Forest 

I.A.l.a.(l) 

Hardvood 

I.A.l. a. (1) (a) 

Cottonwood ( > 302) 

I. A. l.a. (1) (b) 

Aspen ( 7 302) 

I.A.l. a. (2) 

Conifer 

I. A. l.a. (2) (a) 

Ponderosa Pine (7 302) 

I.A.l.a. (2)(b) 

Spruce-fir (7 30 2) 

I. A. l.a. (2) (c) 

Dout?las-fir (7 302) 

I.A.l.b. 

Non-Commercial 

I.A.l.b.(l) 

Oak (7 302) 

I.A.l.b. (2) 

Pinou JuniDer (> 302) 

I.A.2. 

Forest (10-302) 

I.A.l.a. 

Commercial Forest 

I. A. 2. a. (1) 

Hardwood 

I.A.2.a. (1) (a) 

Cottonwood (10-302) 

I. A. 2 .a. ( 1) (b) 

Aspen (10-302) 

I. A. 2. a. (2) 

Conifer 

I. A. 2. a. (2) (a) 

Ponderosa Pine (10-302) 

I. A. 2 . a. (2) (b) 

Soruce-fir (10-302) 

I. A. 2 . a. (2) (c) 

Douelas-fir (10-302) 

I.A.2.b. 

Non-Commercial 

I.A.2.b. (1) 

Oak (10-302) 

I.A.l.b. (2) 

Pin on Juniper (10-302) 

1.3. 

Non-Forest (less than 102 stock) 

1.3.1. 

Rock /Barren 

I.B.2. 

Brush 

1.3 .2. a. 

Willow 

' 1.3. 2. b. 

Mixed 2 rush 

I.B.2.C. 

Sage 

I.B.3. 

Grass 

1.3. 3. a. 

Meadi -w 

I . B . 3 . b . 

Sonoian 

I.B.3.C. 

Montane 

I .B . 3 . d. 

Alpine Xeric 

I.B. 3.e. 

Alpine Mes'c 

II. 

Water 

III. 

Snow 

IV. 

Shadow 


SAN JUAN NATIONAL FOREST 
Ground Cover Type Key - Data Base 
(Cover types are underlined) 
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San 'Juan National Forest 
Ground Cover Type Key - Data Base 

The objective of this key is to classify the total area within the San Juan 
National Forest into one of 25 cover types by three acre cells. 

I. LAND - Cell covered by > 502 land. 

Yes - Go to I. A. 

No - Go to II. 

I. A. FOREST (>102 stocked) - Cells covered by tree species >102 crown 
cover. (Trees are further defined as woody vegetation capable 
of producing a woody stem 2 12 feet in height. This Includes oak and 
other tree species that are < 12 feet in height due to site limiting 
conditions. ) 

Yes - Go to I.A.l. or 2. 

No. - Co to I.B. 

I. A. 1 . FOREST (> 302) - Cell^ covered by tree species >-a>02 crown cover. 

Yes - Go to I.A.l. a. or b. 

No - Go to I. A. 2. 

I.A.l. a COMMERCIAL FOREST - > 502 of the crown cover is one or more of 

the following commercial species: ponderosa pine, spruce-fir, 

Dougias-fir, aspen, or cottonwood. 

Yes - Go to I.A.l.a.(l) or (2). 

No - Go to I A.l.b. 

I. A.l.a.(l) HARDWOOD ->502 of the crovn cover is one or more of the 
following hardwood species: Aspen or Cottonwood. 

I 

Yes - Go to I. A. l.a.(l) (a) or (b). 

No - Go to I.A.l. a. (2). 

I. A.i.a. (1 ) (a) COTTONWOOD ->502 of the crown cover is Cottonwood. 

Yes - The cover type is Cottonwood ( > 302) . 

No - Go to I. A.i.a. (1) . (b) . 

I.A.l. a. (1) (b) ASPEN - > 502 cf the crown cover is aspen. 

Yes - Tie cover type is asren (>302) . 

I. A.i.a. (2) CONIFER - >502 of the crown cover is one or more of the 

following conifer species: ponderosa pine, spruce-fli. , 

or Douglas-fir. 


Yes - Go co I. A.i.a. (2) (a), (b), or (c). 
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I.A.l.a. (2) (a) 
Z.A.l.a. (2) (b) 
I.A.l.a. (2) (c) 
I.A.l.b. 

I.A.l.b. (1) 

I.A.l.b. (2) 

I. A. 2. 

, I. A. 2. a. 

I. A. 2. a. ( 1 ) 


PONDDIOSA PINE - >50Z cf the crown cover is *»-*i*« 

Yes - The cover type is ft.i fine. ( >3QZ) . 

No - Go to I.A.l.a. (2) (b) or (c) . 

SPRDCE-FIR - > 50Z of the crown cover is spruce-fir. 

Yes - The cover type is soruce-f ir ( > 30Z) . 

No - Go to I.A.l.a. (2) (c) . 

DOUGLAS FIR - 7 50Z of the crown cover is mixed Douglas 
fir and white fir. 

Yes - The cover type is Douglas fir (>3QZ) . 

NON-COMMHICIAL - > 50Z of the crown cover is one or more 
of the following non-commercial species: pinon pine, 

juniper, or oak. 

Yes - Go to I.A.l.b. (1) or (2). 

OAK. - >501 of the crown cover is oak. 

Yes - The cover type is oak ( > 30Z) . 

No - Go to I.A.l.b. (2). 

PINON- JUNIPER - >50Z of the crown cover is pinon/ juniper . 

Yes - The cover type is pinon-Tuniper (>302) . 

FOREST (10-30Z) - Cells covered by tree species. 10-3 0Z 
crown cover. 

Yes - Go to I. A. 2. a. or b. 

COMMERCIAL FOREST ->50Z of the crown cover is one or more 
of the following commercial species: ponderosa pine, 

sprucu-f ir , Douglas fir, aspen, or cottonwood. 

Yes - Go to l.A.2.a. (1) or (2). 

No - Go to I. A. 2.0. 

HARDWOOD -?50Z of the crown cover is one or more of the 
following hardwood species: Aspen or cottonwood. 

Yes - Go to I.A.2.a.(l) (a) or (b). 

No-Go to I. A. 2. a. (2). 

COTTONWOOD - 7 50Z of the crown cover is cottonwood. 

Yes - The cover type is cottonwood (10-30") . 

No - Co to I. A. 2. a. (1) (b) . 


I. A. 2 .a. (1) (a) 
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I. A.?.. a. (1) (b) 
I. A. 2. a. (2) 

I. A. 2. a. (2) (a) 

I.A.2.a. (2) (b) 
I. A. 2. a. (2) (c) 
I. A.2.b . 

l.A.2.b. (1) 

I.A.2.b. (2) 
I.B. 

1.3.1. 

I.B. 2. 


ASPEN - ‘>50Z of the crown cover is aspen. 

Yes - The cover type is aspen (10-30%) . 

CONIF^I - >501 of the crown cover is one or aore of the 
following conifer species: ponderosa pine, 3pruce-fir, 

or Douglas fir. 

Yes - Go to I. A. 2. a. (2) (c) , (b), or (c). 

PONDEROSA PINE - >50Z of the crown cover is ponderosa 
pine. 

Yes - The cover type is ponderosa pine (10-30Z) . 

No - Go to I. A. 2. a. (2) (b) or (c) . 

SPRUCE-FIR - >50Z of the crown cover is spruce-fir. 

Yes - T he cover type is soruce-fir (1C-30Z) . 

No-Go to I. A. 2. a. (2) (c) . 

DOUGTAS FIR - >50% of the crown cover is mixed Douglas 
fir and white fir. 

Yes - The cover type is Douglas fir (10-30%! . 

NON-COMMERCIAL - > 50Z of the crown cover is one or more 

of the following ncn- commercial species: pinon pine, 

juniper, or oak. 

les - Go to I. A 2.b. (1) or (2). 

OAK - > 50Z crown cover is oak. 

■^es - The cover type is oak (10-30Z) . 

No - Go to I.A.2.b . (2) . 

PINON- JUNIPER - > 50Z of the crown cover is pinon Juniper. 

Yes - The cover type is pinon-juniper (10-30Z) . 

NON-FOREST (less than 10Z stocking) - Cells covered by 
<10Z crown cover of tree species. 

Yes - Go to I.B. , 1., 2., or 3. 

ROCK/BARREN - <30Z vegetative ground cover is present. 

Yes - Cover type is rock/barren. 

No - Go to As. Z or J. 

BRUSH - > 30Z of the area is covered by brush species. 

Yes - Go to 1.3.2. a, b, or c. 

No - Go to I.B. 3. 
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I.B.2.a. 


I.B.2.b. 


I.B.2.C. 


I.B.3. 


I* 3* 3. a. 


I.B.3.0. 


i. 3. 3 . c. 


I.B.3.d. 


1.3. 3 . e. 


WILLOW (brush) - >302 willow crown cover is present. 

Yes - The cover type is willow . 

No - Go to I.B.2. b or c. 

MTXH) BRUSH - >302 of the area is brush other than 
oalcbrush, willqi^or sage. 

Yes - The cover type is mixed rush . 

No - Go to I.B.2.C. 

SAGE BRUSH - > 302 sage brush crown ‘cover is present. 

Yes - The cover type is sage brush . 

GRASS - >302 of the area is grass and herbaceous plants. 
Yes - Go to I.B.3. a, b, c, d, or e. 

MEADOWS (wet) - The area is dominated by grasses and other 
herbaceous plants requiring constant water availability. 
The elevation range for this cover type is > 5,500 feet. 

Yes - The cover type is meadow . 

No-Go to I.B.3. b, c, d, or e. 

SONORAN GRASSLAND - The area is dominated by grasses and 
other herbaceous plants. The elevation range is 5,500 to 

7.000 feet. 

Yes - The cover type is Sonoran grassland . 

No - Go to I.B.3. c, d, or e. 

MONTANE GRASSLAND - The area is dominated by grasses and 
other herbaceous plants. The elevation range is 6,900 to 

9.000 feet. 

Yes - The cover type is Montane grassland . 

No - Go to I.B.3. d or e. 

ALPINE XERIC - The area is above timberline and dotr lated 
by dry sine grasses and other herbaceous plants. The 
elevation range of this cover type is > 11,000 feet. 

Yes - The cover type la Alpine Xeric . 

No - Go to I.B.3.C. 

ALPINE MESIC - The area is above timberline and dominated 
by wet or moist site grasses and other herbaceous plants. 
The elevation range of this cover type is > 11,000 feet. 

Yes - The cover type is Alpine Mesic . 


33 




II. WATER - Cells covered by > 50Z water. 

Yes - Ground cover tvpe is water . 

TII. SNOW - This cover class division is included because areas will seasonally 
be covered with snov. Landsat will record this information, if present. 
Snow is not a valid ground cover type. 

IV. SHADOW - This cover class division is included because steep topography on 
the Forest produces shaded cells regardless of suci angle. Landsat will 
record this information, if present. Shadow is not a valid ground cover 
type. 
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